Using School Choice Lotteries to Test Measures of School Effectiveness

The measurement of school effectiveness is a central feature of educational accountability policies in all 50 US states and around the world. While school accountability measures are often based on test score levels (e.g., percent proficient), critics argue that test score gains are a fairer way to judge schools’ contributions to student achievement (e.g., Ladd and Walsh 2002, Ryan 2004). Such “value-added” measures (VAMs) have now been introduced into the accountability regimes of at least 30 US states (Blank 2010). The growing interest in VAMs has given rise to a large literature which deals with technical issues such as model specification, choice of sample and outcome, and measurement error in the estimation of “school effects” (Raudenbush and Willms 1995; Meyer 1997; McCaffrey et al. 2003; Rubin, Stuart, and Zanutto 2004; Reardon and Raudenbush 2009). Yet random variation in school attendance is both rare and necessary to test the validity of VAMs, and to guide the selection of models for measuring causal effects of schools. In this paper I use data from a public school choice lottery in Charlotte-Mecklenburg (CMS) to test the validity of school value-added models. Students were guaranteed assignment to their neighborhood school but could apply to attend other schools in CMS, with admission to oversubscribed schools determined by lottery. This yields random assignment to schools, albeit within a self-selected sample of applicants. I estimate a variety of school VAMs, varying the model specification, outcome, and sample on which VAM is calculated. I then use these nonexperimental estimates of “school effects” Using School Choice Lotteries to Test Measures of School Effectiveness †